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Abstract. We study the volume functional on the space of con- 
stant scalar curvature metrics with a prescribed boundary metric. 
f~^ ■ We derive a sufficient and necessary condition for a metric to be a 

critical point, and show that the only domains in space forms, on 
which the standard metrics are critical points, are geodesic balls. In 
rH I the zero scalar curvature case, assuming the boundary can be iso- 

^m. ■ metrically embedded in the Euclidean space as a compact strictly 

I~~J I convex hypersurface, we show that the volume of a critical point is 

,S^ • always no less than the Euclidean volume bounded by the isomet- 

jrt I ric embedding of the boundary, and the two volumes are equal if 

and only if the critical point is isometric to a standard Euclidean 
ball. We also derive a second variation formula and apply it to 
show that, on Euclidean balls and "small" hyperbolic and spheri- 
,._^ ^ cal balls in dimensions 3 < n < 5, the standard space form metrics 

J^ ■ are indeed saddle points for the volume functional. 

a^ 

(N 

t^ . 1. Introduction 

O, 

oo , Given a compact n-dimensional manifold ^ with a boundary S, we 

"^ \ study variational properties of the volume functional on the space of 

constant scalar curvature metrics on Q with a prescribed boundary 

K^ , metric on E. The dimension n is assumed to be > 3. There are several 

H I motivations for us to consider this problem. 

The first motivation comes from a recent result in [2]. There one 
considers an asymptotically flat 3-manifold (M, g) with a given end. 
Let {xi} be a coordinate system at cxd which defines the asymptotic 
structure of {M,g). Let Sr = {x E M \ \x\ = r} be the coordinate 
sphere, where |a;| denotes the coordinate length. Let 7 be the induced 
metric on Sr- When r is large, (5*^, 7) can be isometrically embedded in 
the Euclidean space M^ as a strictly convex hypersurface 5°. Let Vo(r) 
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be the volume of the region enclosed by S^ in M^ and Vr be the volume 
of the region enclosed by Sr in {M^,g). It was proved in |i9j that, as 
r — i> oo, 

(1) V{r) - Voir) = 2mADMT^r^ + o(r') 

whenever uiadm is defined. Here rriADM is the ADM mass of {M^,g) 
[2]. Therefore if R{g), the scalar curvature of g, is nonnegative, then 
by the Positive Mass Theorem [TTJ [22], one has 

(2) V{r) > Voir) 

for sufficiently large r. (Note that the case rriADM = would imply 
{M,g) is isometric to M^, hence showing V{r) = Vo{r).) 

As a statement on volume comparison, ([2]) is rather intriguing. First, 
the curvature assumption it requires is only on scalar curvature. Sec- 
ond, it was formulated as a boundary value problem, i.e. the competi- 
tors involved have a same Dirichlet boundary geometry. It is natural 
to ask whether there exist related results on compact manifolds with 
boundary. 

As a special case of Theorem 13.31 and Theorem 14.41 in this paper, we 
have: 

Theorem 1.1. Let Q be a 3-dimensional compact manifold with a con- 
nected boundary S. Let 'y be a given metric on S such that (S,7) can 
be isometrically embedded in the Euclidean space M.^ as a strictly convex 
hypersurface Sq. Let Ai^ be the space of zero scalar curvature metrics 
g on Q such that the induced metric from g on T, is 'j. 

(i) Suppose g G A^° is a critical point of the volume functional V{-) 
on A^°, then 

V{g) > Vo, 

where Vo is the Euclidean volume of the region enclosed by Sq in 
M^. Equality holds if and only if {Q, g) is isometric to a standard 
Euclidean ball. 
(ii) There exists no element in Ai^ that minimizes volume in A4?^. 

Our second motivation comes from a variational characterization of 
Einstein metrics on a closed manifold |S] [IS]: Let A^_i be the space 
of metrics with constant scalar curvature —1 on a compact manifold 
M without boundary. Then an element (/ G A^_i is an Einstein metric 
if and only if ^f is a critical point of the volume functional V^(-) on 
7V/l_i. This characterization follows from Proposition 4.47 in [5j or 
the argument preceding Lemma 1.2 in [18]. The proof used the fact 
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that Einstein metrics correspond to critical points of the total scalar 
curvature functional 



S{9) = Vigf-^ [ R{g)dV, 

J M 



while the restriction of S'(-) to the space M._i agrees with —V{-)^. 

We want to establish a similar result on compact manifolds with 
boundary, with an aim to therefore find a proper concept of metrics 
that would sit between constant scalar curvature metrics and Einstein 
metrics. 

As a special case of Theorem 12.11 in this paper, we have: 

Theorem 1.2. Let Q be a compact n- dimensional manifold with smooth 
boundary Ti. Let'y be a given metric on S and K = —n{n — l) orO. Let 
M.^ be the space of metrics on Vt which have constant scalar curvature 
K and have induced metric on E given by 7. Then g G AA^ is a 
critical point of the volume functional V{-) on M.^ if and only if there 
is a function X on fi such that A = on S and 

(3) - {AyX)g + V^A - ARic((?) = g, 

where Ag, V^ are the Laplacian, Hessian operator with respect to g and 
Ric{g) is the Ricci curvature of g. 

It will be shown in Theorem 13. 21 that if (7 is a metric, defined on some 
open set, which satisfies ([3]) with some function A, then g necessarily 
has constant scalar curvature. Furthermore, if Q is indeed a closed 
manifold and g is a metric on Q with negative scalar curvature for 
which ([3]) holds with some function A, then g is an Einstein metric. 

Our method in deriving the first variation of the volume functional 
also leads to the following characterization of geodesic balls in the hy- 
perbolic space H" and the sphere S" through the first variation of the 
total mean curvature integral (see Theorem 13. II and Proposition 13.11) . 

Theorem 1.3. Let Q be a connected domain with compact closure in 
H" or S" and with a smooth (possibly disconnected) boundary S. // 
Q C S", we also assume the volume ofQ is less than half of the volume 
o/§". Let g be the corresponding space form metric on Q. Then Q is 
a geodesic ball if and only if 



i H'{0) = 



for any smooth variation {g(t)} of g on Q such that g{t) and g have 
the same scalar curvature and the same induced boundary metric. Here 
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iJ'(O) is the variation of the mean curvature of S in {Q,g{t)) with 
respect to the outward unit normal. 

In particular, Theorem 11.31 implies that if Q is not a geodesic ball 
in H" or the half sphere S", then there exists a deformation g of the 
standard metric g onQ such that g and g have the same scalar curvature 
and the same induced boundary metric on dQ, and 

H< (f H, 

an Jan 

where H, H are the mean curvature of dVL in (fi, g)^ (i7, g). One wants 
to compare this with the results in [T9] and [20j (see the remark after 
Proposition 13. ip . 

In [18], Schoen gave the following conjecture concerning the volume 
functional V^(-) on a closed hyperbolic manifold. 

Conjecture Let {M"',h) be a closed hyperbolic manifold. Let g be 
another metric on M with R{g) > R{h), then V{g) > V{h). 

This conjecture remained widely open until recently its 3-dimensional 
case followed as a corollary of Perelman's work on geometrization [HI 
[T5] . It is natural to wonder if there exists a similar conjecture or result 
on compact manifolds with boundary with a hyperbolic metric. In 
[1], related results were established by Agol, Storm and Thurston, on 
compact 3-manifolds whose boundary are minimal surfaces. In this 
paper, we note that the above conjecture on closed manifolds does not 
generalize directly to manifolds with boundary if only the Dirichlet 
boundary condition is imposed. 

Theorem 1.4. Let H^ be the hyperbolic space and g^i be the standard 
hyperbolic metric on H^ . There exists a small constant 6 > such that 
if B is a geodesic ball in H^ with geodesic radius less than 6, then there 
is another metric g on B such that g induces the same boundary metric 
on dB as g^ does, R{g) = R{gm), and V{g) < V{gm) on B. 

The paper is organized as follows. In Section [21 we first analyze the 
manifold structure of the space of constant scalar curvature metrics 
with a prescribed boundary metric, then we compute the first variation 
of the volume functional and derive the critical point equation. We 
also relate the first variation of the volume functional on domains in an 
Einstein manifold with non-zero scalar curvature to the first variation 
of the total boundary mean curvature integral of the associated metric 
variation. In Section [3l we show that the only domains in space forms, 
on which the standard metrics are critical points, are geodesic balls. We 



Volume functional of compact manifolds with boundary 5 

also establish some general properties for metrics satisfying the critical 
point equation. Then we focus on the zero scalar curvature case to 
prove a theorem that compares the volume of a critical point with the 
corresponding Euclidean volume. In Section HJ we derive the second 
variational formula and apply it to geodesic balls in space forms. In 
particular, we show that, on Euclidean balls and "small" hyperbolic and 
spherical balls in dimensions 3 < n < 5, the volume functional achieves 
a saddle point at the standard space form metrics. For completeness, 
we include an appendix in which we construct traceless and divergence 
free (0, 2) symmetric tensors with prescribed compact support on space 
forms, which are needed in the second variation construction. 

Acknowledgment: The authors would like to thank Robert Bartnik, 
Todd Oliynyk and Andrejs Treibergs for useful discussions. 

2. First variational formula for the volume functional 

Let fl be an ra-dimensional, connected, compact manifold with a 
smooth (possibly disconnected) boundary S. Let 7 be a smooth metric 
on E. Let iS'^'^ be the space of W'^'^ symmetric (0,2) tensors on Q. We 
will always assume k > | + 2 so that each h G S'^''^ is C^'" up to the 

boundary. Let Sq' be the subspace consisting those h with /i|t'(s) = 0. 
That is to say, h{v,w) = for all v,w tangent to S. Let ^A^ be the 
open set in iS'^'^ consisting g which is a Riemannian metric g > such 
that g\T{s) = 7- Let iy^~^'^(fi) be the space of W'^~'^''^ functions on Vt. 
Let 71. be the scalar curvature map which maps g G A^-y to its scalar 
curvature R{g)- The fact that TZisa, smooth map was shown by Fischer 
and Marsden in [8J: 

Lemma 2.1. The map IZ : M.y —>■ iy^~^'^(r2) is smooth. 

Let go G A^-y. Suppose the scalar curvature of go is a constant K. 
We have 

Lemma 2.2. Suppose is not one of the Dirichlet eigenvalues of the 
operator 



in~l)A,, + K. 



Then, near go, the set 



M^ = {ge M,\ n{g) = K) 
is a submanifold of Ai.y. 
Proof. The linearization of 7?. at go is 
(4) DTZg.ih) = ~Ag^^{tTg,h) + divg, {diYg^{h)) - {h, Ric(^o))3o 
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for h in the tangent space of qq in ^A^, which is equal to Sq' . Here 
tig^^ih) and d\Vg^{h) denote the trace of h and the divergence of h with 
respect to the metric ^^o- Let / G W^~'^''^{yt) which is the tangent space 
at an element in W^~'^''^{VL). Consider the following boundary value 
problem: 

\-{n-l)/\g,^u-Ku = / in fi 
In = on S, 

which has a unique solution by the assumption on the Dirichlet eigen- 
values and the Fredholm alternative, see p^. Let h = uqq. We con- 
clude that DTZg^{h) = f. Hence DTZg^ is surjective. As iSq' is a Hilbert 
space, the kernel of DTZgg automatically splits. By the implicit function 
theorem, the lemma follows. D 

We want to apply Lemma 12.21 to domains in space forms: the Eu- 
clidean space MJ^, the hyperbolic space H" and the sphere S*^. 

Corollary 2.1. Suppose Q is a connected domain with compact closure 
in M", M"' or S" and with a smooth (possibly disconnected) boundary 
E. Let g be the standard metric on Q and let '-f = g\^. Then, near g, 
the set JkA.^ is a submanifold of M.^, where (i) K = if Q C M", (ii) 
K = -n{n -l)ifnc W, and (m) K = n{n - 1) if n C S"" and 
V{n) < il^(§"), where V{n) and \/(§") denote the volume of {n,g) 
and S". 

Proof. Cases (i) and (ii) follow immediately from Lemma [2. 2[ In case 
(iii), since the volume of fi C S" is less than the volume of a hemisphere, 
the first Dirichlet eigenvalue for the Laplacian is larger than n by the 
Faber-Krahn inequality |21j , the result follows from Lemma 12.21 again. 

D 

Next we want to consider the volume functional 

V -.M^^R, 

which is a smooth functional on Ai^. For g G Ai^, the first variation 
of V{-) at g is: 

(6) DVgih) = ^ ftTg{h)dVg. 

We are interested in critical points of V{-) restricted to Aif^ , when ^A^ 
is a submanifold. First, we will construct explicit deformations on A^^ 



7 
'< 

7 

K 



Proposition 2.1. Let g^ G AUs be a smooth metric such that the first 
Dirichlet eigenvalue of {n — l)Agg + K is positive. Let h be a smooth 
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symmetric (0,2) tensor on Vt such that /i|t(9C) = 0. Let g{t) = go + th 
which is a smooth metric provided \t\ is small enough. Then there is 
to > and e > such that for all \t\ < to the following Dirichlet 
boundary value problem has a unique solution u{t) such that 1 — e < 
u<l + e: 

, , J aAg(^t)U — R{t)u = —Ku°', in Vt 

\u = 1, on S, 

where a = 4{n — l)/{n — 2), a = {n + 2)/{n — 2), and R{t) is the scalar 
curvature of g{t). Moreover, v = ^\t=o exists and is a smooth function 
on Q (the closure of Q), which is the unique solution of: 

^g) Un-l)A,,v + Kv =^f(0), ^nQ 

]v = 0, on S. 

Proof. Since the first eigenvalue of (n — l)AgQ + i^ is positive, tliere is 
a smootli function on fi and 6 > such that 

(9) {n - 1) A,,^ + K(j) + 6(f) = 

in Q and > on fi, see [10] for example. Let b = minjy^, then 6 > 0. 
Suppose 1 > t > is small enough such that g{t) is a Riemannian 
metric on Q. For each t, let L(-) denote the operator 

aA,(,)(-)-i?(t)(-) + i^(-r- 
Then 
(10) 

L(l + t0) = aA3(i)(l + t0) - i?(t)(l + t0) + K{1 + t0)" 

= aAg^il + t<p) - K{1 + t0) + « (A3(i) - Ag,) (1 + t0) 

- {R{t) -K){1 + tcj)) + K{1 + t(l)f 

< ^(5t0 + Cit + Cst^ 

n — 2 

where Ci is a positive constant depending only on go and h, and C2 > 
depends also on (p. Similarly, 

(11) 

L(l - t0) = aA<,(i)(l - t(j)) - i?(t)(l - t(j)) + K{1 - t(t)Y 

= aAg,{l - t(j)) - K{1 -t<f)) + a {Ag^t) - A^J (1 - t0) 

- {R{t) - K){1 - t(j)) + K{1 - t(t)Y 

> -^i;5t(l) - Cst - Cit^ 
n — 2 
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where C3 is a positive constant depending only on qq and h, and C4 > 
depends also on cf). By rescaling 0, we may assume that (456)/(r2 — 2) > 
2Ci and (4(56)/(n-2) > 2C3. Then for i > small enough L(l+t0) < 
and L(l — t0) > 0. By [161 Theorem 2.3.1], ([7]) has a solution m 
satisfying 1 — t(f)<u<l+t(j) provided t > is small enough. The 
proof for t < is similar. 

To prove uniqueness, let ui and U2 be two solutions of ([7]) such that 
1 — e < Ml, M2 < 1 + e. Then there is (5i > depending only on qq and 
h such that for |t| small enough: 

(12) 

5i j (mi - UifdVgit) < — - / K{ui - UifdVgi^t) 

+ / [K{u\ - ul) - R{t){ui - M2)] (Ml - U2)dV, 



m 




<{C,\t\ + C,e) / {u^-u^ydVg^t) 
Jn 

where C5 and Cq are constants depending only on go and h. Hence if 
e > is small enough and |t| is small enough, we must have ui = U2- 

To prove the last part of the proposition, let u(t), t 7^ be the 
solution of ([7]) obtained above, then l'u(t) — 1| < |t|0. Let w = {u — \)/t. 
Then w satisfies: 

^^,,. =i?(tV-^^-^(4^,inl^ 
^ ' ^ =0, onS. 

Since the right side of the equation is bounded by a constant indepen- 
dent of t and X & VL, there exist /3 > and C7 > independent of t 
such that the Holder norm with respect to Qq with exponent j3 oi w 
in Vt is bounded by C7, see [TT] Theorem 8.29]. By the Schauder esti- 
mates [m Theorem 6.19], one can conclude that for any k > 2, then 
(jk,i3 norm of w is uniformly bounded by a constant independent of t. 
Hence for any tj — > 0, we can find a subsequence which converge to 
a solution t> of ([8]). Since v is unique by the assumption on the first 
eigenvalue of {n — l)Agf^ + K and the Fredholm alternative, we conclude 
that fj|t=o = V. This completes the proof of the proposition. D 

Theorem 2.1. Let g G Ai^ be a smooth metric such that the first 
Dirichlet eigenvalue of {n — l)Ag + K is positive. Then g is a critical 
point of the volume functional in M.^ if and only if there is a smooth 
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function X on Q such that 

-{AgX)g + VlX - XRic{g) = g in Q ; 



^^^^ ^ X = on S. 

Proof. By Lemma [2.21 there is a neighborhood U oi g in Ai^ such that 
U n A4^ is a submanifold. Suppose that g is a critical point of V{-) 
in A4f^ . Since the first eigenvalue of {n — l)Ag + K is positive, by the 
Fredholm alternative we have a smooth function A on 17 satisfying: 

(ax =-^(Air + n),infi 
^ ^ 1 A = 0, on S. 



We want to prove that A satisfies the interior equation in flMj) . Let h 
be a smooth symmetric (0,2) tensor with compact support in Q. Let 
g{t) = g + th, then g{t) G A4^ for small t. For each t, consider the 
following Dirichlet boundary value problem: 

1 ttAg(t)M - R{t)u = -Ku", in Q 
u = 1, on S, 

where a = 4(n — l)/(n — 2), a = (n + 2)/(n — 2) and R(t) is the scalar 
curvature of g{t). For |t| small, the equation has a unique positive 
solution u{t) which is smooth up to the boundary by Proposition 12.11 
Moreover, u^/^'^~'^\t)g{t) is in M^ and is a C^ curve in Uf}M^. Since 
M = 1 at t = 0, 

j^{u''^-'\t)g{t))\,=o = ^^ug + K 

where u' = u'{0). Since g is a critical point of V^(-) in A^:^, by (EI), we 
have 

(17) l^(^^u' + tTg{h)yVg = 0. 

Now by Proposition 12.11 again, u' satisfies: 

aAgu' - R'{0) - Ku' = -aKu', in fi 
u = U, on L. 



-'5 
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Since A = on S, by (ITSll and (TTSl) . we have 
(19) 

^"^ ^ u'dV. = ^^ / u'dV. 



n-2j^'^^ n-lj^ ^ 



a/ I —-uAgWdVn 

f XKu' 



/" /'_^^^ _ Ai?'(0) - (1 - a)K\u'\ dVg 

X (Agitlgh) - diVg {diV g (h)) + {h , Ric(^))3) dVg 



In 

where we have used @ in the last step. Combining this with (TTTl) we 
have 

(20) / A (Agitigh) - diYg (div g (h)) + {h , Ric{g)) g) + iig{h)dVg = 0. 
Jn 

Let / = ii g{h). Let v be the unit outward normal of S. Since / has 
compact support in f2, we have 

(21) / XAgfdVg = [ fAgXdVg + ^ (A/, " /A,) = / f A g XdVg 

Jn Jn Jt, Jn 

where we use ^,y to denote -^ for a smooth function ijj on fi, and 



f22) 



Adivg(divg(/i))= / Xg'^g^^hik-sdVg 
o, Jn 



ki^ 



Xig'^g^^hik,jdVg + i A(div/i)fcZ/^ 
(VjA, h)dVg - (f h{VX, v)+ I A(div/i) 

{VlX,h)dVg 

'n 

where we have used the fact that h has compact support. Combining 
i^-(^, we have: 

(23) = [{h, (AgX) g-VlX + Xmc{g) + g)gdVg. 

Jn 

Since h is arbitrary, A must satisfy the interior equation in flT4l) . 



Volume functional of compact manifolds with boundary 11 

To prove sufficiency, suppose there is a smooth function A satisfying 
f lT^ . let /i G 5q be in the tangent space of g in M.^ . Then h is in the 
kernel of DlZg. Let / = trg(/i) as before. By (jl]) and the computation 
in ([2I])-([22D, we have 

= / A (A,/ - div, (div3(/i)) + (/i, Ric{g))g) dVg 

= [{h,{Ag\)g-Vl\ + \mc{g) + g)gdVg- [ fdVg 
Jn Jn 

^^^^ - (f f\u+ <{ h{V\, u)+i X[U- (div,/i) (z/)] 

= - I fdV, 
= -2DV{h), 

where we have used the fact that A satisfies ( IH1) . /i|T(s) = 0, and A = 
on E. Hence g is a critical point of V{-) in Aii^ . D 

Remark 2.1. (i) From the proof of Theorem \2.1\ one can see that 

under the assumptions on g, g is a critical point of the volume 
functional V{-) m M^ if and only ifV'{Q) = f^V{g{t))\t=o = 
for any smooth variation {g{t)} of g in M.!^ . 
(ii) The differential equation in (HM can be equivalently written as 
DTZ*(X) = g, where DTZ* is the formal L'^-adjoint of DTZg, 
and a weak form of (1141) can also be derived using the infinite 
dimensional Lagrangian multiplier method employed by Bartnik 
in [3]. 

Theorem 12.11 shows that, for a constant scalar curvature metric g to 
be a critical point for V{-) in A4^ , there need to exist a function A 
which satisfies both the interior equation 

(25) - (AgA)^ + VgA - ARic(^) =g onn 

and the boundary condition A|s = 0. (Later in Theorem 13.21 one will 
see that (1231) alone implies that g has constant scalar curvature.) It 
remains interesting to know what the first variation of V{-) in M.!^ 
would be if only (125!) is satisfied but A|s is not necessarily zero. 

Proposition 2.2. Let g G Ai!^ be a smooth metric. Suppose there 
exists a smooth function X on Cl such that 

(26) -{AgX)g + VlX-XRicig) = g on a 
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Let {g(t)} be a smooth path of metrics in A4^ such that g{0) = g and 
g{t) e M^. Then 

(27) jV{gmt=o = £xH'{0), 

where H = H(t) is the mean curvature ofH in {Q,g{t)) with respect to 
the unit outward pointing normal vector v. 

Proof. Similar to (!24l) . we have 

= / A (A,/ - div, (divg(/i)) + (/i, Ric{g))g) dVg 
Jn 

(28) = / {h, (A,A) ^ - V^A + \mc{g) + g),dVg - / fdVg 

Jn Jn 

- J fK+J h{VX, U)+<f X[f,- (diVgh) (z/)] , 
i/ 2-1 V 2-1 O 2-1 

where we let h = g'{0), f = tigh and used the fact DlZg{h) = 0. By 
(l26l). we have 



(29) / fdVg =-(f fX,+ (f h{VX, Z/) + / A [/, - (diYgh) (u)] . 

Jn Jt, Jt, Jt. 

Let VeA be the gradient of A on (E,7) and integrate by parts 

i h{VX,u)= i /i(VsA,z/)+ i h{u,u)X^ 

(30) -^^ ^^ ^^ 

= — f Adiv^X + ffi h{iy,iy)X„, 
Jt. is 

where X is the vector field on S that is dual to the one form /i(z/, ■)|t(s) 

on (S,7) and div^X denotes the divergence of X on (^,7). Plug fl30l) 

in fl29|) . we have 

(31) / fdVg = - / (7, h)^X, + (f X [-diV^X + /, - (diVgh) (u)] . 

Jn Jt. Jt 

Now, let p G S and let {x* | i = 1, . . . , n} be a coordinate chart around 
p in f2 such that {x"^ \ A = 1, . . . , n — 1} gives a coordinate chart on S 
and dn = v- Direct calculation shows 

(diVg/l)„ = hnn-n + l^^hAn-B, 
hnA;B =dB{hnA) — ^ AB^riC ~ ^^B^nn ~ ^nB^iA 
=Xa-B + IlAB^nn — Hfi^CA, 

where 11^^ = (Vg^z^, (9^)^ is the second fundamental form of S. Thus, 
(32) (divg/i)„ = hnn-n + div^X + Hhnn - (n, h)^. 
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On the other hand, we have the following formula of the linearization 
of the mean curvature, see equation (42) in [13] for example: 

(33) H'{0) = hlnn.n + \HKn - (H, h) g - [div^/l - ]^d{ilgh)]n. 

(Note that the sign convention of II in [13] is opposite to the one used 
here.) Hence (l32l) and (133!) imply that 

(34) 2if'(0) = [d(tigh) - di\gh]n - div^X - (II, h)^. 
Therefore, fl3Tl) becomes 

l,h),]. 



(35) / fdVg = - / (7, h),K + / A [2if'(0) + 

Jn JT. JT. 



Finally, by (jlj) and the boundary condition /i|t(s) = 0, we have 
(36) jV{9mt=o = £xH'{0). 

As an application, we have the following: 



D 



Corollary 2.2. Suppose g G A4!^ is a smooth Einstein metric with 
Ric((7) = Kg (so K = nn). Suppose k 7^ 0. Let {g{t)} he a smooth path 
of metrics in M.^ such that g{Q) = g and g{t) G A4^ ■ Then 

(37) jV{gmt=o = -^£H'{0). 

In particular, if the first Dirichlet eigenvalue of(n—l)Ag+K is positive, 
then g is a critical point of the volume functional V{-) in A4^ if and 
only if §^H'{Q) = for any smooth variation {g{t)} of g in Ai^ . 

Proof. If g satisfies Ric((7) = ng, then the constant function A = — 1/k 
satisfies ( !26|) . By Proposition 12.21 (1371) is true. The last part of the 
corollary follows from (!37|) and Theorem 12.11 see Remark 12. l( i). D 

For example, if fi is a domain in S", then 

dt n — 1 Jy; 

for any smooth variation {g{t)} of the standard metric on Q which 
keeps the induced boundary metric fixed. 



14 Pengzi Miao and Luen-Fai Tarn 

3. Critical points in space forms 

In this section, we shall discuss the volume functional on domains in 
space forms. 

Theorem 3.1. Let Q be a connected domain with compact closure in 
M", H" or S" and with a smooth (possibly disconnected) boundary E. 
If Q G §", we also assume that V{Q) < |\^(§"). Let g be the standard 
metric on Q and let 7 = 5'|t(e)- Suppose g is a critical point of the 
volume functional V{-) in M.^ , where (i) K = if Q G M", (ii) K = 
-n{n - 1) zfnc W, and (Hi) K = n{n - I) if Vt C. §", then Vt is 
a geodesic ball. Conversely, if Q is a geodesic ball, then the standard 
metric g is a critical point of the volume functional V{-) in M.^ . 

Proof. By Theorem 12. ![ (fi, g) is a critical point if and only if there 
exists a smooth function A such that 

-(A,A)(7 + V^A - ARic(^) = ^ in n ; 
A = on E. 



Taking trace of the equation, we have: 
(38) 



n 



if fsT = 0; 



/\g\ = -^^-^{XK + n) = { \n~^^, iiK=-n{n-l) 



-\n - ^, ii K = n(n - 1] 

n— 1' ^ > 



Hence A satisfies 
(39) 

XK+1 



V^A = ARic(^) - '-^^^g ={ {X-^)g, iiK = -n{n - 1) 



^ " ' n 



{-^-^)9^ ifK = n(n-l). 
In case (i), where fi is a domain in M", (!39|) directly implies 

1 " 

(40) ^ = -777 T-|x|' + Vfo.x' + c, 

where x^ ,■ ■ ■ , x" are the standard coordinates on M" and 6j, c are con- 
stants. By translating the origin, we may assume that A = ~ 2(ra-i) I^P+ 
c for a possibly different c. Since A is zero at the boundary of Vt, Vt 
must be a Euclidean ball and c = 277^371 -R^, where R is the radius of 
the ball. Conversely, if O is a Euclidean ball of radius i?, then it is 
easy to see that A given above satisfies the conditions in Theorem 12.11 
therefore the standard metric is a critical point. 

Next we consider case (ii), where ^2 is a domain in H". Suppose 
the standard metric is a critical point of the volume functional. Let 



Volume functional of compact manifolds with boundary 15 

A satisfy ( l39l) . then A is not identically zero. Since A = on S, there 
must be an interior point p & Q such that VA(p) = 0. Henceforth, we 
use V to denote the covariant derivative with respect to g. 

Embed H" in M"'-*^, the Minkowski space with metric dx\ + - ■ ■ + dx\ — 
dt^ such that 

H" = {(xi,...,a;„,t) \xl + --- + xl~t^ = -l, t > 0} 

and such that p is mapped to the point (0, ■ ■ ■ ,0,1). Then v = 
Er=i^i^ + tf is normal to W with (z/, z/) = -1 in W'^. Let V 
and D be the covariant derivatives of H" and M"'^ respectively. Con- 
sider a point in H" and a function / defined near that point in M"'^. 
Let {cj} be a basis for the tangent space of H", we have 

ViV,/ = DiDjf + %(grad/, u) 

where grad/ is the gradient of / in M"'^, 11^^ = —{De^Cj, v) is the second 
fundamental form of H" and is equal to the induced metric gij on H". 
Let / be the function at+ ^^ where a is chosen so that a+ ^^ = A(p). 
Hence f{p) = X{p) and V/(p) = 0. Since (grad/, z/) = at, one can check 
that / satisfies fl39|) . So V^(A — /) = (A — f)g. Consider a geodesic 
a{s) on EI*^ emanating from p. Restricted to a"(s), the function A — / 
satisfies the ODE 

(A-/r = A-/, 

where " ' " is taken with respect to s. Since initially A — / = (A — /)' = 0, 
A — / must be identically zero. So A = /. But / = A = at the 
boundary of Q. Hence fi is a geodesic ball. When restricted on H", 
t = coshr where r is the geodesic distance from the point (0, ■ ■ ■ , 0, 1). 
Hence a = — ((n — 1) coshi?)~ where R is the radius of the geodesic 
ball n. 

Conversely, if il is a geodesic ball with center at (0, ■ ■ ■ , 0, 1) with 
geodesic radius R. Then 

(41) A ' ^^ ^°^^^ 



n — 1 \ cosh R 



satisfies the conditions in Theorem 12. H which implies the standard 
metric is a critical point. 

Finally we consider case (iii), where fi is a domain with compact 
closure in §". Suppose the standard metric is a critical point of the 
volume functional. Let A satisfy fl5^ . then A is not identically zero. 
Since A = on S, there must be a point p E fl such that VA(p) = 0. 
Embed S" in M"+^ = {{xo,xi, . . . ,Xn)} as the unit sphere centered 
at the origin such that p is mapped to the point (0, ■ ■ ■ ,0,1). Let 
/ = axn — j;^, where a is chosen so that a — ;^^ = \{p). Hence 
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f{p) = \{p) and V/(p) = 0. As in case (ii), one can check that / 
satisfies f l5Ul) . Then one can prove as before that A = /. Since / = A = 
on the boundary of Q, Q must be a geodesic ball centered at (0, ... , 0, 1) 
whose boundary can not be the equator {x„ = 0}. As we assume 
V{Q) < |y(S"), fi must be strictly contained in the upper hemisphere. 
When restricted to S", x„ = cosr where r is the geodesic distance from 
the point (0, ■ ■ ■ ,0,1). Hence a = {{n — l)cosR)~ where R is the 
radius of the geodesic ball Q. 

Conversely, if f2 is a geodesic ball in S" with radius R ^ ^, then 

/ N , 1 / cos r 

42 A = ( 1 

^ ^ n-1 \cosR 

satisfies flT4l) in Theorem 12.11 If furthermore Q is contained in a hemi- 
sphere, then Theorem 12.11 shows that the standard metric is a critical 
point. D 

As a direct application of Theorem 13.11 and Corollary 12.21 we have 

Proposition 3.1. Let Q be a connected domain with compact closure 
in H"' or S" and with a smooth (possibly disconnected) boundary S. // 
Q C S", we also assume that V{Q) < |V^(§"). Let g be the standard 
metric on Q, let K be the constant that is equal to the scalar curvature 
of g and let 7 = 5'|t(s)- Then Q is a geodesic ball if and only if 



i H'{0) = 



for any smooth variation {g(t)} of g in J^^ . Here H'{0) is the variation 
of the mean curvature of 12 in {fl,g{t)) with respect to the outward unit 
normal. 



Proof. By Theorem 13.11 fi is a geodesic ball if and only if the standard 
metric g is a critical point of the volume functional V{-) in Aif^ . On the 
other hand, as g is an Einstein metric with non-zero scalar curvature. 
Corollary 12.21 shows g is a critical point of V{-) in Aif^ if and only if 



/ H'{0) = 



for any smooth variation {g{t)} of g in Aits . The proposition follows. 

D 

Before we proceed to discuss properties of general critical metrics of 
V^(-), we want to relate the result in Proposition 13.11 to the results in 
[T9] and [20] . Let S be any given compact strictly convex hypersurface 
in M^. If {^,g) is a compact Riemannian 3-manifold with nonnegative 
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scalar curvature whose boundary is isometric to S and has positive 
mean curvature H, then it was proved in [12] that 



(43) f Ho> f H, 

JY, Jan 

where Hq is the mean curvature of S in M'^. In particular, if Q is the 
domain enclosed by S in M'^, fj43|) then implies that 



f H'{0) = 

for any smooth variation {g{t)} of g in Ai^. This contrasts sharply 
with Proposition 13.11 by which we know the unique compact convex 
surfaces in M^ or §■]_ with that property are geodesic spheres. As a 
result, Proposition 13. II implies that (H5]) does not generalize directly to 
an arbitrary compact convex surface in M.^ or H^. On the other hand, it 
was proved in [20] that (H3|) does hold if E is a geodesic sphere in H^ and 
{Q, g) has scalar curvature no less than —6, which is consistent with 
Proposition 13.11 It remains an interesting question to know whether 
(143|) is true for a compact 3-manifold {Q, g) whose boundary is isometric 
to a geodesic sphere in T,^ and whose scalar curvature is greater than 
or equal to +6. 

Next we discuss some general properties of a Riemannian metric g 
for which there exists a function A satisfying the differential equation 
in flT^ in Theorem 12. 1[ 

Theorem 3.2. Let {^,g) be a connected, smooth Riemannian mani- 
fold. Suppose there is a smooth function X on Q such that 

(44) - (A,A)^ + VjA - ARic((7) = g. 

Then 

(i) g has constant scalar curvature. 

(ii) IfQ is compact without boundary and g has negative scalar cur- 
vature, then g is an Einstein metric. 

(iii) IfQ is compact with a smooth (possibly disconnected) boundary 
S such that A = at T,, and if the first Dirichlet eigenvalue of 
{n — l)Ag -\- K is nonnegative where K is the scalar curvature 
of g, then along each connected component Eq, o/ S, the mean 
curvature of Sq, with respect to the outward unit normal v is 
a positive constant. In fact, Sq is umbilic and its second fun- 
damental form IIq, satisfies IIq = aag\T{T.a) for some constant 
an > 0. 



(46) A^A = -- — - - - — ^A 
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(iv) Under the same assumptions as in (Hi), at each point in T,, 

n — 2 
(45) 2Ric(z/, u) + Rj: = K+ -H^, 

where R^, is the scalar curvature of S and H is mean curvature 
ofE. 

Proof, (i) can be proved as in [8]. By taking the divergence of (jS]), and 
using the Bianchi identity, we conclude that XdR = where R is the 
scalar curvature of g. At a point p where X{p) ^ 0, we have dR = 0. 
Suppose p G f2 is an interior point where X{p) =0. The equation 

n R 

n — 1 n — 1 

then implies AA(p) < 0. Thus, either VA(p) 7^ or p is a strict 
local maximal point for A. In either case, we would have dR = in a 
neighborhood around p. Hence, dR = in fi. 

To prove (ii), we know that i? is a negative constant by (i) and the 
assumption. As Q is compact without boundary, it follows from fH6l) 
and the maximum principle that A must be a constant. Hence, g is an 
Einstein metric. 

To prove (iii), we note that the boundary condition A = at S, 
together with (jB]), implies 

(47) V^A = l-g 

^ n — 1 

at S. Now choose a local orthonormal frames {cj} at the boundary so 
that Ci is tangential for 1 < i < n — 1 and e„ = i^ is the unit outward 
normal. For 1 < z < n — 1, fHTI) implies 

7 = VgA(ei,ei) 

77,-1 '^ 

= eiCi^X) - Ve,ei(A) 
= -(Ve,ei,en)e„(A). 
Summing over l<'i<r7, — 1, we have 

. , <9A 

(48) -l^H-, 

where H is the mean curvature of S with respect to i/. If A < 
somewhere, then the set U = {x & Q\X < 0} is a nonempty open set 
contained in Q such that A = at dU because A = at S. Since 
{n — 1)AA + KX = —n < 0, we have 



/ {{n-l)\VX\^-KX'^)dV <Q, 
Ju 
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contradicting the fact that the first Dirichlet eigenvalue of {n—l)A + K 
is nonnegative. Therefore A > in fi. This together with fHS]) imphes 
f^ < at S. Hence if > and e„ = z/ = -VA/|VA|. 

Next, let X and Y be two arbitrary vector fields on S which are 
tangential to S. At E, P7j) implies 

0=Vl\{X,u) 

=Xu{X) - Vxz^(A) 

= -X(|VA|) 

where we used A = at S, and 

^—g{X,Y)=VlX{X,Y) 

n — 1 " 

=XY{\) - VxF(A) 

= - |VA|n, 

where II denotes the second fundamental form of S with respect to 
z/. Hence, along each connected component Sq, of S, we conclude that 
|VA| is a positive constant and II equals a positive constant multiple 
of the induced metric on S. (The constants may depend on a.) 

(iv) follows directly from the Gauss equation and (iii). This com- 
pletes the proof of the theorem. D 

Theorem 13 ■ 1 1 shows that the standard metrics on geodesic balls (which 
are contained in a hemisphere for the case of 5") in space forms are 
critical points of the volume functional in the corresponding spaces of 
metrics. It is natural to ask whether they are the only critical points 
with that boundary condition. Using Theorem [221 we give some partial 
answer to this question. 

Corollary 3.1. Let Q be an n-dimensional compact manifold with a 
smooth connected boundary S. Let '-/ be a given metric on S and let 
K = or —n{n — 1). Suppose g G Ai!^ is a smooth metric and g 
is a critical point of the volume functional V{-) in Ai^ . Let v be the 
outward unit normal vector to S in {^,g). 

(i) If K = 0, (S, 7) is isometric to a geodesic sphere in M" and Q is 
spin if n > 8, then Ric(z/, u) is a non-positive constant along S, 
and Ric(z/, u) = if and only if {Q, g) is isometric to a standard 
ball in R". 
(ii) Ifn = 3,K = 0,Qis oriented and Ric{i>, u) = along S, then 
{^,g) is isometric to a standard ball in M^. 
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(iii) Ifn = 3, K = —6 and (S, 7) is isometric to a geodesic sphere in 
M.^, then Ric(z/, u) is a constant satisfying Ric(i/, z/) < —2 along 
S, and Ric{v,v) = —2 if and only if {^,g) is isometric to a 
geodesic ball in H^. 

Proof, (i) As (S, 7) is isometric to a geodesic sphere, say Eq, in M."', i?s 
is a constant. Hence, by (iii) and (iv) in Theorem I3.2[ Ric{v, v) is a 
constant along S. On the other hand, applying the Gauss equation to 
Eq in M", we have 

(49) R^ = ^Hl 

n — 1 

where Hq is the mean curvature of Sq in M" (which is a constant). 
Hence, it follows from (iv) in Theorem 13. 2[ (I49p and the fact K = 
that 

■n — 2 

(50) 2Ric(z/, u) = —^ \H^ - Hi] . 

n — 1 '- -' 

By the results in [191 [12] , which are generalizations of the positive mass 
theorem in [T71 [22l [18], we have H < Ho, and H = Hq ii and only if 
{^,g) is isometric to a standard ball in M". From these, (i) follows. 

(ii) By the assumption, E is a connected orientable 2-surface. Let 
Kj^ be the Gaussian curvature of (E,7). It follows from (iii), (iv) in 
Theorem 13.21 and the facts K = 0, Ric{v, i/) = that 

which is a positive constant. Therefore, (E, 7) is isometric to a round 
sphere in R^. Now (ii) follows from (i). 

(iii) The proof is similar to the proof of (i). As (E, 7) is isometric to 
a geodesic sphere, say Ei, in H^, R-^ is a constant. Hence, Riciy, u) is 
a constant. On the other hand, applying the Gauss equation to Ei in 
H^, we have 

n — 2 

(51) 2(-2) + i?s = -6 + -Hi 

n — 1 

where Hi is the mean curvature of Ei in H^ (which is a constant). 
Hence, it follows from (iv) in Theorem 13.21 (ISTj) and the fact K = —6 
that 

(52) 2[Ric(^, u) - (-2)] = -[H'- H',] . 

n — 1 

By Theorem 3.8 in [20], we we have H < Hi, and H = Hi ii and only if 
{Q, g) is isometric to a geodesic ball in H^. From these, (iii) follows. D 
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As another application of Theorem 13.21 assuming (S,7) can be iso- 
metrically embedded as a convex hypersurface in M", we can compare 
the volume of any critical point in A^° with the Euclidean volume en- 
closed by (E,7) inM". 

Theorem 3.3. Let {Q, g) be an n-dimensional smooth compact Rie- 
mannian m,anifold with zero scalar curvature, with a smooth connected 
boundary E, such that g is a critical point of the volume functional in 
A4° where 7 = g\Y,- Suppose (S, 7) can be isometrically embedded in M" 
as a compact strictly convex hypersurface Sq. // the dimension n > 8, 
Q is also assumed to be spin. Then 

V{g) > Vo 

where V{g) is the volume of {Q, g) and Vq is the Euclidean volume of 
the domain bounded by Sq in M". Moreover, V{g) = Vq if and only if 
{^,g) is isometric to a standard ball in M". 

Proof. As (7 is a smooth critical point of the volume functional in A1° 
by Theorem 12.11 there is a smooth function A on fi such that A = at 
E and 

-iAgX)g + VlX-Xmcig) = g 

in Q. As g has zero scalar curvature, the condition in (iii) in Theorem 
13.21 is satisfied, therefore the mean curvature H oi H in [Q, g) with 
respect to the unit outward normal z/ is a positive constant. Moreover, 
by dH]), if and IJ satisfies 













-1 




at S. 


Integrating 


on 


s. 


we 

^1 


have 


Js du 


(53) 










= — 


H f AgXdVg 



"" -HV{g), 



n — 1 
where the last step follows from the fact that 

A,A = -. 

n — I 

On the other hand, by the results in |19j . 



(54) f Ho> (h H = \J:\H 
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where Hq is the mean curvature of Sq with respect to the outward 
normal in M". By a Minkowski inequahty [B]: 

(55) |S|2 > -^Vo (f Ho. 

It follows from ([53]), (El) and ((551) that 

ISI n 



-H 



V{g) n-1 
n 

ISI 



(56) < ,^ ,M^i f Ho 

'So 



< 

- Vo 
Hence V{g) > Vq. If V{g) = Vq, then fl54|) becomes an equality. By 
the results in [19] again (see also [12j), we know that {^,g) is isometric 
to a domain in M". Finally by Theorem 13. 11 we conclude that {^,g) is 
isometric to a standard ball in M". 

D 

4. Second variational formula for the volume functional 

In this section, we will compute the second variation of the volume 
functional V{-) at critical points in A4f^ . First, we give a formula for the 
second derivative of the scalar curvature. We remark that our notation 
convention for the curvature tensor gives 

R'ijki = t^idikdji — gudjk) 
in the case that g has constant sectional curvature k. 

Lemma 4.1. Let {g(t)} be a smooth path of C"^ metrics with g[0) = g. 
Let R{t) be the scalar curvature of g{t). Then 

R"iO) = A,i\h\l) + 2{h, VlitTgh))g - 4(V3(diV,/l), h)g 

ld{tTgh)\l-l\ 

+ 2h'PRikpsh"' + Dngih'), 

where h = g'{0), h' = g"{0), Vg(-) and ' ; ' denote covariant derivative 
with respect to g, Rijki is the curvature tensor of g, and D7lg(-) is the 
linearization of the scalar curvature map TZ at g. 

Proof. The first derivative of the scalar curvature is given by 
(58) R'{t) = -A,{tTgg') + dwg{dwg{g')) - {g\ Ric),. 



(57) - 2|div,/i - -d{tTgh)\l - -\Vgh\l - g'^'h^' ,,h,,u 
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For any fixed point p, let {xi} be a normal coordinate chart at p with 
respect to g{0) = g. We will use ',' to denote partial derivative and ';' 
to denote covariant derivative. At t = 0, we have 

-[A,(tr,^')]' = - W' {tT,g'),,-g'%{tT,g'),,)]' 



- g''g''K, {tigh) „. - /\g\h\l + A,(tr,/i' 



"PQ y^a'"! ,ij 



(59) 



- 1^9'^ a'"^ {hjm + hii-j - hij,i) (trg/i)^^ 
+ {(livgh,d{iigh))g " -\d{iigh)\l, 



(60) 



[div,(div,^')]' = {9''9'')'h,i,,. + g'^g'' {g'^,J 

= -{Vgdivgh, h)g - g''g'^g''h„h,i,,, + g^^g'' {g'^,^,^)' 
= -2{VgdiYgh, h)g + h'^^Rikpsh'" - h^'g'^'R.khu 



where we have used the Ricci identities. Now 



ij kl ( I \' ij kl \ / I \ -ps I -ps I -ps I 1 ' 

9 9 {9jl;ki) —9 9 [{9jl;k),i ~ '^ ij9sl;k ^ '- il9js;k ^ '- ik9jl;s\ 

y,i;k - in,)' hsi - {tw h, 



-9''9''' 



and 



(?v((ry'/^ 



^V(/^;7j, = div,(div,/.'), 
1 



= ~l9''9''{9'"'hk^.,,hsi). 



h 



kj;m) iT'sl 



-AMI 



-(?V((nj'M 



7:9^'' 9 {9^"^ {hkm;l + hml;k — hkl-m) hjs) ^ 

{VgdWgh,h)g + -{VlitTgh),h)g 



- \divgh\l + -{divgh,d{trgh)) 



gi 
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-g'^g''^ mYhsi-k = --g'^g^^g""^ {him;j + hmj;i - hij.^) h,i,k 
= -\diygh\l + - {diY gh,d{tTgh))g, 

-g'^g''^ {T'J hjs;k = -^g'^g'^^g""' ihim;l + hm^ - hu;m) hjs;k 

-g'^g^^ (r-J' hji., = --g'^g^^g'"" {him;k + hmk;i - hik-m) hji,^ 

= -9''h''-M^k + \\Vgh\l 
Hence, 

(div, {dwgg'))' = - HVgdwgh, h)g + h'^Rikpsk^'' ' k^'g^'R^kki 

(61) + diV,(diV, h') - ^Ag\h\l + liVlitTgh), h)g 

+ (diV,/l, d{ilgh))g - 2\diVg h^ " ^^^^ Zl^^ ^^ Zl,,; fc • 

Next, 

.g2) -[{g',mc)gr = -{g^^g'^g',,R,,y 

= 2h'^g'''R,khu - {h', mc)g - g'^g^'KkR',1, 



1 
T 



g^^g hikR'ji = -g''^g hikg^'' {hip.jg + hjp-ig — hji-pg — hpg.jl) 



+ g'^g''^g'"^hikhip,jg 

= -\Ag{\h\l) + l\^,h\l-^-{Vl{tTgh),h)g 

+ {VgdiYgh, h)g + h'^g'^R^kki - h'mikpsh'\ 
where we have used the Ricci identity in the last step. Hence 

-[(^',Ric),]' = W'g^'R^khu - {h\Ric)g + ^Aj/^lj - \\Vgh\ 
(63) ^ 4 2 



+ -(Vj(tr,/i), h)g - (Vgdiv,/i, h)g + h'^Rikpsh'". 

So ([57D follows from ([58]), ([59D, dM]) and ([63]). D 

Now we are in a position to compute the second variation of V{-) in 
Ai!^ . We state the formula in a general setting which does not require 
the manifold structure of J^!^ . 
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Theorem 4.1. Let Q be an n- dimensional connected compact manifold 
with a smooth boundary S. Suppose g is a smooth metric on Q such 
that there is a smooth function X on Q satisfying 



(64) 



-iAg\)g + V^A - \Ric{g) = g m Vl ; 
X = on S. 



Let 7 = 5'|t(s) (ind let K be the constant that equals the scalar curvature 
of g. Suppose {g{t)} is a smooth path of metrics in M.^ with g{Q) = g 
and git) G A4^ ■ Let V{t) be the volume of {Q,g(t)), then 



(65) 
V"{0) 



-{tighY + X 



\diy,ih)-^ditTgh)\l + ^\Vgh\l 



dVa 



+ / A [(V.div,/., h)g - {h, VlitTgh))g] dVg 
Jn 

- [l-X [\diygh\l + h^mk^ish'^] dVg, 

where h = g'{0). 

Proof. We first note that V'{0) = by flM|) and the proof of Theorem 
I2.1[ To compute V""(0), we start with 



(66) 



V"{0) 



l{tTghf-^\h\l +^{tTgh') 



dVr^ 



gi 



where h = g'{0) and h' = g"{0). Our aim is to express the last integral 
in terms of h. 

Recall that (IMl) implies 



(67) 
and 
(68) 



A,A 



V^A = ARic 



n — 1 



n 



(KX + n) 



iXK + l)g. 



Applying the fact that g{t) has constant scalar curvature K and Lemma 
14.11 we have 



(69) 



=R'\0) 
=1 - AgitTgh') + diVgidiYgh') - {h', Ric)g, 
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where 

/ =Ag{\h\l) + 2{h, Vl{tTgh))g - 4(V,diVg/l, h)g 

(70) - 2\div,{h) - ^d{tT,h)\l - ll^Ml - g^'h^'.^^hi,,, 

+ 2h^Rikpsh^ ■ 

Note that / involves only h and its derivatives. In what follows, we will 
omit the volume form and the area form in integrals for convenience. 
All integrals are taken with respect to the metric g = g{0). Integrating 
by parts, we have 

[-\Ag{tTg{h')) + XdiYgidiYgh') " X {h' , Ric) g] 

[-{tTgh')iAgX) + {VlX,h')g - X {h' , Ric) g] 
(71) 

+ i [X.,{tlgh')-h\u,VX)] 



/ tr gh' + f X.uitrgh') - f /i'(z/, VA) 



where we have used A|s = 0, fl67l) and fl68|) . The condition A|s = 0, 
together with /i'|r(s) = 0, also implies 

-X,^{tigh') + /i'(z/, VA) = on S. 

Combining this with fp^ and fl7T]) . we have. 



(72) / itigh') = - XL 

Jn Jn 

Next we compute the integral of XI. Integrating by parts and apply- 
ing (1^ and (^E\i . we have 

[ XAgi\h\l) = [ {AgX)\h\l - f XMl 

/^o\ Jn Jn Jt. 

^ {KX + n)\h\l- i X.^h^ 



n^ ^ Jt, 



Integrating by parts, we also have 
(74) 



Wh^-.,h 



I {Xg^'^h'^hi,,,,),, - g^"^X,XK,k - Xg^^h'^'h 
Jn 



f -gP'iX,^h'%^,k - Xg^'h^'hi^.kp. 
Jn 
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Now 
(75) 
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((7^^A;p/i'^/i,,);fc - g^'^\pkh"'hi, - 9'"'X;ph''.^,h 



Iq 



= i f'^X.,,h"'hi, + / ^— {KX + 1) \h\l - [ Xg^'^Rpkh^'hig - / X,p{div gh)ih'P 

= (f X.,,\h\l + f -^ (KX + 1) \h\l - f XgP'^Rpkh'%, - f A;p(div,/i);/i'P 
Jt, 7q ^ ~ -L Jn Jn 

where we have used the fact that /i|t(e) = 0. Also 

(76) 

/ X,,{diYgh)ih'P = [ [X{diYgh)ih'P],, - X{diYgh)i,ph'P - X\diYgh\l 

Jn Jn 

= - X{VgdiYgh,h)g- / X\diYgh\l. 
Jn Jn 

By the Ricci identify, 

(77) 

XgP'^h"'hig,kp = I X{VgdiYgh, h)g + Xg^mpkh^'hi, - Xh^^kpish^K 



It follows from ^-^ that 



Xg^''h'\Ai,,k = - i X.JW^ 



n 



E ' Jnn-l 



{KX + l)\h\l 



(7J 



- / 2X{VdiYgh,h)g- I X\d\Ygh\ 
Jn Jn 



+ / Xh'^Rkpish"'. 



Combining ([TOD, ([73]) and dZHD we have 



XI 



(79) 



\h\l + 2X{h, Vl{ilg{h)))g - 2X{Vd\Yg{h), h), 



2X\diYg{h)-]^d{iTgh)\l+\x\Vh\l 



+ / X\diYMl + Xh''PRkr,lsh'K 



'9"'\g 



Now ([65]) follows from ([66]), ([72]) and (1791). 



D 
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Next we focus on domains Q in space forms on which the standard 
metrics are critical points of V{-). By Theorem l3.lt those domains are 
precisely geodesic balls. 

Theorem 4.2. Let Q be a geodesic ball with compact closure in MJ^, H" 
or S^. Let S be its boundary and g be the standard metric on Q. Let 
7 = gWcE) O'lT'd K be the constant that equals the scalar curvature of g. 
Let {g{t)} be a smooth path of metrics in M.!^ with g{Q) = g. Let h 
denote g'{0) which is assumed to be nonzero. 

(i) //f] C M" ornc SI, then 

^,Vigmt=o>0 

for any h satisfying 

divgh = and tigh = 0. 

(ii) If Q G H", then for any p E Q, there is a constant 6 > 0, 
depending on p and {fi,g), such that 

^,V{gmt=o>0 

for any h which has compact support in Bsijp) C Vt and satisfies 
divg/i = and iigh = 0. 

Here Bs{p) denotes the geodesic ball with radius 6 centered at p 

inW. 

Proof. From the proof of Theorem 13. ![ we know there exists a smooth 
positive function A on satisfying fl64|) . (The explicit expression of 
A is given by (jlOD, (jH]) and (02])). Applying Theorem |0 and the 
assumption div gh = and tigh = 0, we have 

72 



(80) ^^WI*=o = 11 ^I^^^IS -II ^h'^Rkpish^'- 

As g has constant section 
integral above reduces to 



As q has constant sectional curvature ,^ -.^ , the integrant in the last 

^ n(n— 1) ' <=' 



Xh'PRkpish"' = .^^^. [{tighf - \h\l] . 
^ nin — 1) '- ^-^ 



Therefore (I80p becomes 



(81) !Lvm._„ = \j^x\vMi^^J^^jm 
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where we again used the assumption tigh = 0. As A > on fi, (i) 
follows directly from fIST]) . 

Now suppose Q C Wl, we then have K = —n{n — 1) and 



d\^, ., 1 /• ,,„ ,,2 1 

n\ . ^.,,, , 



(82) ^vW|..„ = jy^A|V,/.|^|y^A|/, 



For any p G ^2, first choose 5 > such that -^^(p) C ^2 and 

min A > -A(»), max A < 2A(»). 
If h has compact support contained in Bs = Bs{p), then 

(83) ^ ""' ""' 



1 

'bs " JBs 



>m{i; I MhUll- / \h\l 



where we used \'Vgh\g > \V\h\g\g in the second step. Let Xi{B) be the 
first eigenvalue of A^ on Bs, then 



B4) ^^wi*=o ^^(^) {^^i(^^) - 1} y^ 1^1 



<f 



Aslim^^o Ai(i?5) = + CO, we conclude that -^V{t)\t=o > if 5 is smaller 
than some constant ^o depending only on p and {^,g). Therefore, (ii) 
is proved. D 

Lemma 4.2. Suppose g G Ai^ is a smooth Einstein metric satisfying 
the property that the first eigenvalue of [n — 1) A^ + K is positive. Let 
h be a symmetric (0,2) tensor on Vt such that /i|t(s) = 0, tigh = and 
diygdivg{h) = 0. Then there is a variation {g{t)} C Aii^ of g such that 
g'iO) = h. 

Proof. As in the proof of Theorem 12. 11 we can find g(t) G Ai^ such that 

4 

g{t) is of the form u{t)-^''^{g + th) with u(0) = 1. Moreover v = u'{0) 
satisfies: 

^g5) Un-l)AgV + Kv ={^)Dngih),mn 

]v = 0, on S. 

As tTg{h) = 0, divg{divg{h)) = and g is Einstein, we know DTZg{h) = 
by (!58|) . As the first eigenvalue of {n — l)Ag + K is positive, we have 
f = 0, hence g'{0) = h. D 
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In the appendix following this section, we will explicitly construct 
trace free and divergence free (0, 2) symmetric tensors with prescribed 
compact support on space forms (see also [H [7]). Therefore, by the 
existence of such tensors and by Theorem 14.21 and Lemma 1^^ we have: 



Corollary 4.1. Let {^,g) be given as in Theorem \4.2\ There exists a 

dt 



variation {g{t)} C AA^ of g such that ^V {g{t))\t=Q = and 



^Vigmt=o>0. 

In particular, the volume of the standard metric g is a strict local min- 
imum along such a variation. 

Next, we show that there indeed exist deformations along which the 
volume of the standard metric is a strict local maximum. 

Theorem 4.3. Let Q be a geodesic ball with compact closure in M", 
H" or §" . Let S be its boundary and g be the standard metric. Let 
7 = gWcE) o,nd K be the constant that equals the scalar curvature of g. 
Suppose the dimension n satisfies 3 < n < 5. 

(i) IfVtC. M", there exists a variation {g{t)} C AA^ of g such that 



''V{g{t))\t=o = ^ and 

V{g{t)%=,<Q. 



dt 

dt^ 



(ii) If Q G H" or S", there exists a small positive constant 6 such 
that, if the geodesic radius of Q is less than 6, then there exists 
a variation {g(t)} C M.^ of g such that ^V{g{t))\t=Q = and 

^V{g{t))\t=,<Q. 

Proof. We first consider the case fi C M". We use go to denote the 
standard Euclidean metric on M" and Vo(-) to denote the covariant 
derivative taken with respect to g^. Let h be an arbitrary, nonzero, 
symmetric (0, 2) tensor that is parallel on ($7, ^fo) and satisfies tr^g/i = 0. 
Define h = \h. Then /i|r(s) = and satisfies 

(86) tig^h = 

(87) {diygoh)i = g^^Xkk], 

(88) div,,(div,„/i) = (Vj„A,/i),„ = 0, 
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where we have used the equation 

(89) VlX = -^^0. 

By Lemma [4. 2[ h G Tg^AA^ . Plug this h into fl65l) in Theorem 14. II and 
use the fact Qq has zero curvature, we have 



^.ym)\.-«-l 



X 

n 



oldiv<;o(^)lj + 7lVo/i|'o + {^0{diYg,h),h)g, 



2 i"--yu\--/i9 ' ^1 

Applying ( 1871) and (1891) . integrating by parts and using the fact that h 
is parallel, we have 

(90) / Mdiv,Mi =W7Z^ I >^"\h\l- 

Jn 



(^1' /^|^"|» =2(^731) y^^^i"!- 



1 



(92) / X{divg,h),,,h^^ = - —^ I A>| 

Thus, 



2| £.12 

n 



n-1 '- ' '^° 



1 fn — 6 

1 / \^\h, 

1 90 



(93) ^"(o)=^(;r^j.4^'l^l-<0' 



for n = 3,4, 5. 

Next, we consider the case Q C H"'. For any k > 0, consider the 

metric Qk = (l — xl-^l^) S'O' 'which is defined on {|a;| < ^} and has 
constant sectional curvature — k^. Let B = {\x\ < 1} and S be its 
boundary. Let A^ be the smooth function on B defined by 

^'') ^'==(;7^('-IT^^°^^"^ 

where r is the geodesic distance from the origin in {B,gi^). We also 
define 

(95) Ao = — ^ (1 - |xp) . 

For K > 0, the function A^ satisifes 

(Q,a\ / -(A<;,A^)^« + Vg^A«-A^Ric(^^) = g^ 

^ ^ 1 AJs = 0, 
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and the metric g^ is a critical point of the volume functional V^(-) on the 

manifold A^^" , where 7^ = 5'k|t(S)- For any h G Tg^Ai^^ , 

define the second variational functional of the volume 



(97) 



dlVgM - 7^d{tTgMl + jl'^Ml 



dV. 



JB 

- ^ \\M^YgMldV, + j^ yA. [iiTg^' - \h\l] dV^ 

according to fl65|) in Proposition 14. 11 where Vk(-) denotes the covariant 
derivative with respect to (7^ and dVf^ denotes the volume form of g^. 

Now, for K = 0, choose an h^ G Tg^^M.^^^ such that J-'o{ho) < 0. 
The existence of such an h^ was proved in (i). For any n G (0,1], 
by Proposition 12.11 there is a t^ > and e^ > such that for all 
\t\ < t^, gn{t) = gK + tho is a smooth metric on B and the following 
Dirichlet boundary value problem has a unique solution UK{t) such that 
1 - Ck < u^{t) < 1 + e«: 

faA^4i)M-i?«(t)M =-Kul, in B 
\^Uf, = 1, on S. 

where a = 4(n - l)/(n - 2), a = (n + 2)/{n - 2), and i?«;(t) is the 
scalar curvature of gnit). Moreover, v^ = ^'^(0) exists and is a smooth 
function on B which is the unique solution of: 

[Vf, =0, on S, 

where 

R'M =Dng^iho) 

= - Ag^{tig^ho) + diVg^(diVg^/io) - {ho,Ric{g^))g^. 

Since g^ -^ go in C°^{B) as k ^ 0, we have DTZg^iho) -^ in C°°{B) 
and, by dniD, t^K ^ in C°^{B). Now define c/«(t) = ur^{t)g^{t), 
{9^{t)}\t\<t^ is a smooth path in M^^^'"''^^'" with ^(^^(O) = g^. Let 
^« = fl'«(0), then 

4 

(100) /ik = 7:Vf,go + ho. 

n — 2 
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Let K — i> 0, we have /i^ -^ ho in C°°{B) and J-'nihi^) -^ J^o^ho). As 
ho is chosen so that J-'o{ho) < 0, we see that there is a small kq > 0, 
depending only on _B, go and /iq, such that J^^{h^) < for k < Kq. 
Recall 

we conclude that the case VL C H" is proved by scaling the metric (?« to 
a metric with constant sectional curvature —1. 

The case fi C §" can be proved in a similar way by replacing g,^ with 

/ ^2 \ -2 
(101) g^=[l + -\x\^\ go 

and by replacing A^ with 

(102) ^*=(;r^(-i + J^"-'' 

where r is the geodesic distance from the origin in {B,gf^). This com- 
pletes the proof of the theorem. D 

As a corollary of Theorem I3.H Theorem 13.31 and Theorem 14.31 we 
have the following nonexistence result on the global volume minimizer 

inA^o. 

Theorem 4.4. Let Q be a domain in M" bounded by a smooth, compact, 
strictly convex hypersurface S. Let go be the standard Euclidean metric 
on M" and let 7 = 5'o|t(s)- If ^ is a round sphere in R", the dimension 
n is assumed to satisfy 3 < n < 5. Define f3 = mi{V{g) \ g G M.^}, 
then there does not exist a smooth metric g in 7V1° such that V{g) = (3. 

Proof. Suppose there exists a smooth metric g G A^° with V{g) = /?, 
then g is a. critical point of V{-) in A^°. By Theorem 13.31 V{g) > Vq, 
where Vq is the Euclidean volume of Q. Therefore, /3 > Vq. 

Suppose S is not a round sphere in R". By Theorem 13. ![ go is not a 
critical point of V{-) in Ai^. In particular, there is a path of metrics 
{g{t)} in M° with ^(0) = go such that f^V{g{t))\t=o < 0. Hence, 
V{g(t)) < Vo for small positive t, contradicting /5 > Vq. 

If S is a round sphere in R", by Theorem 14.31 there is a path of 
metrics {g{t)} in M^ with g{0) = go such that ■^V{g{t))\t=o = and 
^V{g{t))\t=o < 0. Hence, V{g{t)) < Vo for small t, again contradicting 
/? > K)- The theorem is proved. D 

Before we end this section, we give a discussion of "large" geodesic 
balls in S" which strictly contains a hemisphere. 
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Proposition 4.1. Let Q be a geodesic ball in S" with geodesic radius 
R satisfying ^ < R < n. Let g be the standard metric on S". Let S be 
the boundary of Q and 7 = 5'|t(e)- Suppose {g{t)} is a smooth path of 
metrics in AA^ with g{fi) = g and g{t) E A^" . Then 



d 



^V{g{t)%=o = ^. 
If h = g'{0) is nonzero and satisfies 

divgh = and tigh = 0, 
then 

^Vigmt=o<0. 

Proof. We embed §" in W^'^^ as the unit sphere 

S = 1(^X0, Xi, • • • ; Xn) I 3^0 + Xi + . . . + X„ ^ i J. 

Suppose Q is given by the set 

fi = S'^ n {xn > -} 
a 

for some constant a < —1. Consider the function 

A = {axn - 1) . 

n — 1 

Then g and A satisfy 

r -(A^A)(7 + V2A-ARic((7) =gmn; 
^ ^ \ X = on S. 

From the proof of Theorem 12.11 we know -^V{g(t))\t=Q = 0. By Theo- 
rems?!] and the assumption diVgh = and tigh = 0, we have 

(104) ^^(^W)l*=o = i / ^l^^l' + ^ ^I^IS- 

Note that, unhke the case f2 C S" in which A is positive, we have A < 
in this case. Therefore, ■^V{g{t))\t=o < 0. D 

Comparing Proposition 14.11 with the case f2 C S" in Theorem 14.21 
we find there exists a dichotomy between the variational properties of 
V{-) on "small" geodesic balls in §" strictly contained in a hemisphere 
and "large" geodesic balls in S" strictly containing a hemisphere. 
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5. Appendix: TT tensors with prescribed compact support 

In this appendix, we give a construction of trace free and divergence 
free (0, 2) symmetric tensors with prescribed compact support on a 
rotationally sjTiimetric manifold. As mentioned earher, such tensors 
are used to construct special metric variations in Section |H 

Let m > 2 be an integer. Let S"^ be the m-dimensional sphere with 
the standard differential structure. Let g^m be the standard metric on 
S*™ with constant sectional curvature +1. For a given R > 0, consider 
the product manifold 

(105) M=(0,i?)x5™ 
with a rotationally symmetric metric 

(106) g = l-dr^ + r^g^^, 

where r denotes the usual coordinate on (0, R) and N = N{r) is a given 
smooth positive function on (0, R). 

In what follows, we let u denote points in S"^ and let Y = Y{u!) be 
a fixed eigenfunction of S™ = (S'™,(y'§m) whose eigenvalue n is not the 
first eigenvalue of S™', i.e. 

(107) ^gsmY + kY = 0, K>m. 

Theorem 5.1. For any < ri < r2 < R, and any given smooth 
function a{r) with support in (ri,r2), there exists a smooth divergence 
free and trace free (0, 2) symmetric tensor h on (M, g) such that the 
support of h is contained in (ri,r2) x S"™ and h{dr,dr) = a{r)Y . 

Proof. We first assume that a{r), b{r), c{r), d{r) are some smooth func- 
tions of the variable r alone, which are to be determined later. For 
each r, let T,r be the leaf {r} x S"^ in M. We define a (0, 2) symmetric 
tensor h on M as follows, 

(108) h{dr,dr)=a{r)Y, 

(109) h{dr,-)\T(j,^) = b{r)dY, 

(110) hi; OItce.) = r'[c{r)Vl^Y + rf(r)r^s™], 

where /^(^r, ■)|r(s^), h{- , ■)\tcs^) are the restriction of h{dr,-), h{-,-) to 
the tangent space of S^, dY is the differential of Y on 5™ and V^^^F 
is the Hessian of Y on S™. 
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Let {uja I A = l,...,m} be a local coordinate chart on S"^ and 
Oa = -^j- be the associated tangent vector. Then 

tigh =g''''h„ + g^'^hAB 

(111) =N'^a{r)Y + c{r)Ag^^^Y + md{r)Y 

= [N'^a{r) — Kc{r) + md{r)]Y. 

To compute div^/i, we let (9„ = Nd^ be the unit normal vector to E,.. 
Let {r*j!j} be the Christoffel symbol of g with respect to the frame field 
{Oa, dn}, where z, j, A; G {1, . . . , m, n}. Let ';' denote the covariant dif- 
ferentiation with respect to g and ',' denote the partial differentiation. 
Direct computations shows 

(diVg/l)„ =hnn;n + g^^ Ka;B 

^iT'nn^n + fi' \l>'nA,B ~ "-jaI nB ~ '^ni'- Ab) 

(112) =hnn;n + g \hnA,B — hcA^nB ~ hnC^ AB ~ hnn^AB) 
= hnn;n + fi' [hnA.B — hnC^ AB ~ ^CA^^B + hnnM-AB) 
= hnn;n + diV2j/l(9„, OI^S.)] " (/i|r(E,)> I^) + hnnH, 

where divE,,(-) denotes the divergence operator on (Sr,5'|T(s^)), Uab = 
(Va^(9„, 9b) is the second fundamental form of S,. and H is the mean 
curvature of S^. Similarly, 

{diVgh)A =hnA;n + g^'^hAB;C 

(113) =hnA;n + fi' \hAB,C — Kb^^c ~ ^AiX^Bc) 



--hnA;n + (dWj^Mn^r)]) A + g""^ Kb^AC + Hh 



nA- 



Meanwhile, by flT06D . flTOTl) . flTOHj) . flT09|) and flTTOj) . we have 

• H = Nmr-^, IIab = Nr~^gAB 

• h^A = Nh{dr,dA) = Nb{r)YA 

• Kn = N^h{dr,dr) = N^a{r)Y 

• Kn;n = /i„n,n = Ndr[N^a{r)Y] = Ndr[N^a{r)]Y 

• KA;n = KA,n-hmT\^-KATl^ = iVa, [iV6(r)F^] -iVV-i6(r)F^ 

• (/i|t(eJ,II) = Nr-^[-t^c{r)+md{r)]Y 

• g^^'KBllAc = N^r~'b{r)YA 
and 

(114) divsj/i(5„, OItcs.)] = r-Mwg^4Nb{r)dY] = -r-^Nh{r)KY, 

div2,[/i|T(E.)] =divgs,„ [c(r)V3^„F + rf(r)F^§™] 

(115) =c(r)[d(A<;g^r) + (m - l)rfr] + d(r)rfr 

={c(r)[-K + (m - 1)] + rf(r)}(iF, 
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where in (11151) we have also used the fact Ric{gs^) = (m — 1)(7§™. 
Therefore, it follows from fillip . flll2p and (11131) that h is trace free 
and divergence free if and only if the following system holds 

{N'^a — KC + md = 

dr{N^a) — r'^bn — r^^[~Kc + md] + mr~^N'^a = 
Ndr{Nb) + c[-K + {m-l)] + d + mr-^N% = 0, 

or equivalently 

—Kc + md = —N'^a 

(117) { r-%K = dr{N^a) + {m + l)r-^N^a 



-K + (m 



l)] + d = -NdriNb) - mr-^N%. 



Now, let a = a{r) be given as in the Theorem. By the second equation 
in (I117p . b = b{r) is then determined accordingly, furthermore b{r) is 
smooth and has compact support in (ri, r2). With a{r) and b{r) given, 
the first and the third equations in (I117P become a linear system for 
c = c{r) and d = d{r). As 



;ii8) 



—K, m 

-K + im — 1) 1 



{m — 1)(/t; — m) 



and K is chosen so that k > m, c{r) and d{r) are uniquely deter- 
mined by the first and the third equations in (I117P and they both are 
smooth and have compact support in (ri,r2). With such a choice of 
a{r) , b{r) , c{r) , d{r) , we conclude that the (0,2) symmetric tensor h, 
defined by (I108p - (lll0p . satisfies all the conditions in the Theorem. D 
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